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Abstract 


Measures  of  the  cognitive  complexity  of  leaders  have  been  used  to  infer  the  flexibility,  open- 
endedness,  and  information- orientation  of  their  decision-making  in  international  and  non-state 
confrontations.  At  present,  there  are  two  major  methods  of  “assessment  at  a  distance”  used  in  this 
context.  One  uses  computer  scoring  to  develop  personality  profiles  of  leaders;  the  other  uses  a 
more  labour-  and  time-intensive  human  scoring  system  to  track  changes  in  the  target’s  thinking  to 
predict  the  outcome  of  a  particular  confrontation.  If  computer  scoring  were  able  to  make  event- 
specific  predictions,  the  saving  in  time  and  work  would  be  substantial.  This  study  compared  the 
two  systems  to  establish  (a)  whether  the  computer-scored  system  could  replace  human  scoring, 
and  (b)  using  the  example  of  the  South  Ossetia  War  between  Georgia  and  Russia,  which  method 
was  a  better  predictor  of  rising  and  falling  tension.  The  data  confirmed  the  relevance  of 
integrative  complexity  measurement  in  a  new  context,  that  of  an  ongoing  confrontation  with 
changing  levels  of  tension,  up  to  and  including  war,  between  a  major  and  a  minor  national  power. 

The  correlation  between  scores  from  the  two  methods  was  low;  at  high  levels  of  cognitive 
complexity,  it  was  essentially  zero.  The  human  scoring  of  integrative  complexity,  which  tracks 
changes  in  complexity  over  the  duration  of  a  particular  event,  was  closely  tied  to  the  course  of  the 
confrontation;  the  computer  scoring  of  cognitive  complexity,  which  profiles  complexity  as  a 
stable  personality  characteristic,  was  not.  Thus,  although  computer  scoring  has  significant 
advantages  in  cost  and  time,  it  does  not  accomplish  the  same  goals.  This  finding  has  important 
implications  for  security/intelligence  applications,  as  the  computer  scoring  approach  is  an 
attractive  way  to  process  large  amounts  of  information.  However,  our  data  indicate  that  the 
negative  trade-off  between  speed  and  accuracy  is  serious  enough  to  opt  for  the  more  laborious 
human  tasking  if  the  goal  is  the  prediction  of  crisis  outcome. 


Resume 


Nous  avons  eu  recours  a  des  mesures  de  la  complexite  cognitive  chez  les  leaders  pour  prevoir  le 
caractere  conciliant,  ouvert  et  axe  sur  Tinformation  du  processus  de  prise  de  decision  dans  les 
situations  d’affrontement  intemationales  et  non  etatiques.  A  Theme  actuelle,  on  utilise 
principalement  deux  methodes  «  devaluation  a  distance  »  dans  ce  contexte.  La  premiere  etablit 
un  profil  de  la  personnalite  des  chefs  en  ayant  recours  a  la  cotation  informatisee;  la  deuxieme  fait 
appel  a  une  methode  de  cotation  par  des  humains  plus  exigeante  en  main-d’ceuvre  et  en  temps 
pour  effectuer  un  suivi  des  changements  dans  le  raisonnement  du  sujet  cible  afin  de  prevoir 
Tissue  d’un  affrontement  en  particulier.  Si  la  cotation  informatisee  permettait  de  faire  des 
previsions  relativement  a  un  evenement  precis,  T  economic  de  temps  et  de  travail  ainsi  realisee 
serait  considerable.  Dans  le  cadre  de  la  presente  etude,  nous  avons  compare  les  deux  systemes 
dans  le  but  de  a)  determiner  si  le  systeme  de  cotation  informatise  pouvait  remplacer  le  systeme  de 
cotation  par  des  humains;  b)  determiner  quelle  methode  permettait  le  mieux  de  prevoir 
Texacerbation  et  le  relachement  de  la  tension,  a  Taide  de  Texemple  du  conflit  Georgie-Russie  en 
Ossetie  du  Sud.  Les  donnees  ont  confirme  la  pertinence  de  la  mesure  de  la  complexite  integrative 
dans  un  contexte  nouveau,  celui  d’un  affrontement  en  cours  entre  une  puissance  nationale 
majeure  et  une  puissance  nationale  mineure,  affrontement  caracterise  par  un  niveau  de  tension 
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changeant  qui  peut  atteindre  et  comprendre  la  guerre. 

La  correlation  entre  les  scores  obtenus  a  l’aide  des  deux  methodes  etait  faible;  a  des  niveaux 
eleves  de  complexite  cognitive,  elle  etait  essentiellement  egale  a  zero.  La  cotation  de  la 
complexite  integrative  par  des  humains,  qui  permet  de  suivre  les  changements  sur  le  plan  de  la 
complexite  pendant  le  deroulement  d’un  evenement  donne,  correspondait  etroitement  a 
1’evolution  de  l’afffontement;  ce  n’etait  pas  le  cas  de  la  cotation  informatisee  de  la  complexite 
cognitive,  qui  presente  la  complexite  comme  un  trait  de  personnalite  stable.  Par  consequent, 
meme  si  la  cotation  informatisee  presente  de  nombreux  avantages  sur  le  plan  des  couts  et  du 
temps  requis,  elle  ne  permet  pas  d’atteindre  les  memes  objectifs.  Ce  constat  a  des  incidences 
importantes  sur  les  applications  en  matiere  de  securite/renseignements,  etant  donne  que  la 
cotation  informatisee  est  une  fa9on  commode  de  traiter  un  volume  important  de  donnees. 
Neanmoins,  nos  resultats  indiquent  que  le  gain  de  rapidite  est  accompagne  d’une  diminution  de 
l’exactitude  suffisamment  importante  pour  justifier  le  recours  a  la  methode  plus  laborieuse  faisant 
appel  aux  humains  lorsque  l’objectif  est  de  prevoir  Tissue  d’une  crise. 
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Executive  summary 


The  Complexity  Construct  in  Political  Psychology:: 
Personological  and  Cognitive  Approaches 

Peter  Suedfeld;  DRDC  Toronto  CR  2010-022;  Defence  R&D  Canada  -  Toronto; 
February  2010. 

Introduction  or  background:  The  analysis  of  the  cognitive  processes  of  leaders  has  been  a 
useful  tool  in  the  prediction  of  crisis  outcome  in  international  and  non-state  confrontations. 
Because  leaders  of  nations  or  violent  non-state  groups  are  rarely  if  ever  available  for  first-hand 
psychological  studies,  such  forecasts  are  typically  made  “at  a  distance.”  Quantitative  methods  for 
these  assessments  can  be  categorized  as  either  profiling,  whose  goal  is  to  generate  a  multi¬ 
dimensional  picture  of  the  individual’s  personality  that  in  turn  can  be  used  to  predict  his  or  her 
behavioural  tendencies  in  general,  or  as  tracking,  a  method  that  uses  repeated  measures  of 
changes  in  the  state  of  a  particular  variable  as  a  specific  situation  unfolds  to  predict  future 
decisions  related  to  that  situation. 

Cognitive  complexity,  a  measure  of  flexible,  open-ended,  information-oriented  thinking  and 
decision-making,  has  been  used  in  both  profiling  and  tracking  modes  to  predict  international 
crisis  outcomes.  Both  approaches  use  content  analysis  of  spoken  and  verbal  outputs  of  relevant 
leaders  as  indices  of  their  level  of  complexity.  The  question  of  whether  one  mode  is  a  more 
reliable  predictor  than  the  other  is  an  important  one  in  the  assessment  of  adversarial  intent;  it  is 
made  even  more  salient  by  the  fact  that  the  most  widely  used  profiling  system,  Margaret  G. 
Hermann’s  Profiler  Plus  software  measuring  conceptual  complexity  (CC),  is  a  much  less  labour- 
intensive  method  than  the  most  widely  used  tracking  system,  Peter  Suedfeld’s  manual  for  hand¬ 
scoring  situation-related  changes  in  integrative  complexity  (IC).  If  the  two  methods  yield  similar 
results,  abandoning  the  hand-scoring  system  would  facilitate  research  in  this  area. 

Results:  The  current  study,  carried  out  by  Suedfeld’s  research  group  with  the  cooperation  of 
Margaret  Hermann,  correlated  scores  derived  by  the  two  methods.  Texts  originally  scored  by  the 
IC  system  were  re-scored  using  Profiler  Plus,  and  vice  versa.  A  low  but  reliable  correlation  was 
found  at  low  levels  of  complexity.  At  medium  and  high  levels,  the  relationship  eroded, 
eventually  to  zero.  Analysis  of  a  third  sample,  tracking  the  statements  of  Georgian  President 
Saakashvili  before  and  during  the  2008  Georgia-Russia  War  over  Abkhazia  and  South  Ossetia, 
found  that  IC  scoring  showed  a  close  relationship  to  rising  and  falling  tensions  between  the  two 
states;  CC  scoring  showed  no  such  relationship. 

Significance  In  view  of  the  vast  amount  of  information  that  must  be  rapidly  accessed,  analyzed, 
and  understood  in  developing  intelligence  forecasts,  it  would  be  highly  desirable  if  computerized 
CC  scoring  could  replace  the  IC  scoring  method.  However,  the  data  obtained  in  this  study  show 
that  only  IC  scoring  closely  monitors  the  level  of  tension  and  therefore  usefully  assesses  the 
probability  of  violence  during  a  specific  international  crisis.  However  attractive  rapid  computer 
scoring  may  appear,  the  trade-off  in  predictive  power  would  make  such  a  substitution  inadvisable. 

Future  plans:  The  next  Subproject,  now  being  completed,  applies  IC  scoring  and  other  methods 
of  thematic  content  analysis  (TCA)  to  the  prediction  of  conflict  course  and  outcome  to  four 
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confrontational  situations:  The  Russian-Georgian  War  (including  the  scoring  of  statements  by 
Russian  President  Medvedev).  Domestic  political  violence  in  Zimbabwe,  terrorist  attacks  by  al 
Qa’ida,  and  the  international  confrontation  concerning  Iran’s  nuclear  program  (scoring  the  leaders 
of  Iran,  the  USA,  and  Israel). 
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Le  concept  de  complexity  en  psychologie  politique  :  approches 
fondees  sur  I’etablissement  de  profil  et  sur  le  suivi 

Peter  Suedfeld;  RDDC  Toronto  CR  2010-022;  R  &  D  pour  la  defense  Canada  - 

Toronto;  Fevrier2010. 

Introduction  ou  contexte  :  L’ analyse  des  processus  cognitifs  des  leaders  s’est  averee  un  outil 
precieux  pour  predire  Tissue  d’une  crise  dans  les  situations  d’affrontement  intemationales  et  non 
etatiques.  Etant  donne  que  les  leaders  des  nations  ou  des  groupes  non  etatiques  violents  sont 
rarement,  voire  jamais,  disponibles  pour  des  etudes  psychologiques  directes,  ces  evaluations  sont 
habituellement  realisees  «  a  distance  ».  On  distingue  deux  categories  de  methodes  quantitatives 
utilisees  pour  ces  evaluations.  La  premiere,  T  etablissement  de  profil  {profiling ),  a  pour  objectif  de 
brosser  un  portrait  multidimensionnel  de  la  personnalite  du  sujet,  portrait  qui  peut  etre  utilise  pour 
predire  ses  tendances  comportementales  en  general.  La  deuxieme,  le  suivi  {tracking),  fait  appel  a 
des  mesures  repetees  du  changement  d’etat  d’une  variable  donnee,  a  mesure  qu’une  situation 
particuliere  evolue,  pour  prevoir  les  decisions  futures  liees  a  cette  situation. 

Pour  prevoir  1’ issue  de  crises  intemationales,  on  a  eu  recours  a  la  complexite  cognitive  -  aussi 
bien  en  mode  «  etablissement  de  profil  »  qu’en  mode  «  suivi  »  -  une  mesure  du  raisonnement  et 
du  processus  de  prise  de  decision  conciliant,  ouvert  et  reposant  sur  1’ information.  Les  deux 
approches  font  appel  a  une  analyse  de  contenu  de  la  production  orale  et  verbale  des  leaders 
pertinents  comme  indicateurs  de  leur  niveau  de  complexite.  Dans  T  evaluation  de  l’intention 
antagoniste,  la  question  de  savoir  si  l’une  des  deux  methodes  est  un  predicteur  plus  fiable  que 
l’autre  est  un  aspect  important;  elle  est  d’autant  plus  cruciale  que  le  systeme  d’ etablissement  de 
profil  le  plus  largement  utilise,  le  logiciel  Profiler  Plus  de  Margaret  G.  Hermann,  mesurant  la 
complexite  conceptuelle  (CC),  est  une  methode  beaucoup  moins  exigeante  en  main-d’ceuvre  que 
le  systeme  de  suivi  le  plus  utilise,  le  guide  de  cotation  manuelle  des  changements  lies  a  la 
situation  sur  le  plan  de  la  complexite  integrative  (Cl)  de  Peter  Suedfeld.  Si  les  deux  methodes 
permettent  d’obtenir  des  resultats  analogues,  l’abandon  de  la  cotation  manuelle  faciliterait  les 
recherches  dans  ce  domaine. 

Resultats  :  La  presente  etude,  effectuee  par  le  groupe  de  recherche  de  Suedfeld  avec  la 
cooperation  de  Margaret  Hermann,  a  mis  en  correlation  les  scores  obtenus  a  l’aide  des  deux 
methodes.  Les  textes  qui  avaient  deja  fait  l’objet  d’une  cotation  initiale  a  l’aide  du  systeme  Cl  ont 
ete  cotes  de  nouveau  au  moyen  du  systeme  Profiler  Plus,  et  vice  versa.  Une  correlation  faible 
mais  fiable  a  ete  observee  aux  faibles  niveaux  de  complexite.  Aux  niveaux  moderns  et  eleves,  la 
relation  diminuait,  pour  finalement  atteindre  zero.  L’analyse  d’un  troisieme  echantillon,  qui 
consistait  en  un  suivi  des  declarations  du  president  georgien  Saakashvili  avant  et  apres  le  conflit 
entre  la  Georgie  et  la  Russie  en  2008  concemant  l’Abkhazie  et  l’Ossetie  du  Sud,  a  revele  que  les 
scores  obtenus  avec  la  methode  Cl  correspondaient  plus  etroitement  a  l’exacerbation  et  au 
relachement  de  la  tension  entre  les  deux  Etats;  on  n’observait  pas  une  telle  correspondance  avec 
la  methode  de  cotation  CC. 

Portee  de  Particle  :  Compte  tenu  du  volume  considerable  d’information  qui  doit  etre  consultee, 
analysee  et  interpretee  rapidement  pour  effectuer  des  previsions  fondees  sur  des  renseignements, 
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il  serait  extremement  souhaitable  de  pouvoir  remplacer  la  methode  de  cotation  Cl  par  la  methode 
CC  informatisee.  Toutefois,  les  donnees  obtenues  dans  la  presente  etude  montrent  que  seule  la 
cotation  Cl  permet  de  suivre  de  pres  le  niveau  de  tension  et,  du  meme  souffle,  d’evaluer  de 
maniere  utile  la  probabilite  de  violence  durant  une  crise  intemationale  donnee.  Si  attrayante  que 
soit  la  methode  rapide  de  cotation  informatisee,  un  tel  remplacement  ne  serait  pas  judicieux  car  il 
se  solderait  par  une  diminution  de  l’efficacite  predictive. 

Projets  futurs  :  Le  prochain  sous-projet,  maintenant  acheve,  applique  la  cotation  Cl  et  les  autres 
methodes  d’analyse  du  contenu  thematique  (ACT)  pour  predire  revolution  et  Tissue  d’un  conflit 
dans  quatre  situations  :  le  conflit  entre  la  Russie  et  la  Georgie  (notamment  la  cotation  des 
declarations  du  president  russe  Medvedev);  la  violence  politique  interne  au  Zimbabwe;  les 
attentats  terroristes  d’Al-Qa'ida  et  Topposition  intemationale  au  programme  nucleaire  de  TIran 
(par  la  cotation  des  declarations  des  leaders  de  TIran,  des  Etats-Unis  et  d’lsrael). 
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Introduction 


The  assessment  of  leader  personalities  and  decision  patterns  has  been  an  important  component  of 
political  psychology  even  before  the  field  acquired  a  label  and  identity  of  its  own.  In  the  very 
first  Handbook  of  Political  Psychology  (Knutson,  1972),  at  least  three  of  the  16  chapters  —  by 
Knutson,  Glad,  and  Katz  —  were  to  a  great  extent  focused  on  reviews  of  publications  linking  such 
characteristics  with  the  political  ideology  and  behaviour  of  a  diverse  list  of  individuals  and 
groups.  “Assessment  at  a  distance”  became  one  of  the  hallmark  activities  of  the  field,  several  of 
whose  major  contributors  combined  innovative  methodological  and  theoretical  ideas  with 
practical  applications  for  diplomacy  and  negotiating  strategy. 

1.1  Cognitive  Style  as  a  Personality  Variable 

Among  the  best-established  and  most  frequently  used  variables  in  the  study  of  the  personality- 
politics  link  have  been  authoritarianism  (Adorno  et  al.,  1950)  and  its  variants,  such  as  dogmatism, 
rigidity  (Rokeach,  1960),  and  “right-wing  authoritarianism”  (RWA;  Altemeyer,  1981).  The 
original  version,  whose  stated  goal  was  to  identify  people  who  would  be  susceptible  to  Fascist 
ideology,  was  explicitly  content-oriented,  assuming  that  for  the  most  part  (although  not 
universally)  high  levels  of  authoritarianism  would  be  found  on  the  right  wing  of  the  political 
spectrum.  The  trait  itself  was  theoretically  the  outgrowth  of  child-rearing  and  societal  ideologies, 
and  was  manifested  in  respect  for  parents  and  traditional  authorities,  unwillingness  to  change  or 
even  to  examine  one’s  ideas,  ethnocentrism,  and  derogation  of  outgroups  whether  ideological  or 
racial. 

The  authoritarian  personality  was  conceived  as  a  Gestalt  incorporating  cognitive,  emotional,  and 
unconscious  components.  Later  approaches,  however,  concentrated  more  on  stable  ways  of 
thinking,  labelled  cognitive  styles.  Among  the  earliest,  the  concepts  of  dogmatism  and  rigidity 
(Rokeach,  1 960)  presented  a  reaction  to  the  political  bias  of  the  authoritarianism  work,  proposing 
a  theory  and  scales  that  dealt  with  closed-mindedness  and  inflexible  thinking  regardless  of  any 
specific  political  position.  RWA,  despite  its  label,  is  also  content-free,  at  least  in  conception:  it  is 
defined  by  acceptance  of  the  conventional  mores  of  one’s  society,  obedience  to  its  leaders,  and 
punitiveness  toward  those  who  dissent  from  either.  Obviously,  such  a  pattern  can  exist  in  both 
conservative  and  liberal  or  radical  political  systems. 

1.2  Cognitive  Complexity 

Another  construct,  cognitive  complexity,  has  similarities  and  overlaps  with  some  aspects  of 
authoritarianism,  but  even  more  so  with  dogmatism,  rigidity,  and  a  number  of  later  cognitive  style 
formulations  such  as  need  for  cognition  (Cacioppo  &  Petty,  1982),  orientation  toward  uncertainty 
(Sorrentino  et  al.,  1992),  and  need  for  closure  (Kruglanski  &  Webster,  1996).  Although  cognitive 
complexity  has  been  defined  and  measured  in  a  number  of  ways  by  different  theorists  and 
researchers  (see,  e.g.,  Goldstein  &  Blackman,  1978;  Schroder  &  Suedfeld,  1971),  some  common 
threads  run  through  all  of  these  versions.  In  general,  high  complexity  is  thought  to  be  marked  by 
openness  to  and  searching  for  new  information;  the  ability  to  modify  or  even  abandon  problem¬ 
solving  strategies  or  decisions  if  the  new  information  indicates  that  they  are  inappropriate; 
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tolerance  for  high  information  load,  uncertainty,  and  lack  of  closure;  the  integration  of  decisions 
with  each  other  both  temporally  and  across  domains;  the  perception  of  nuances  and  fine 
distinctions;  and  a  willingness  to  consider  alternative  and  opposing  viewpoints. 

As  the  above  list  implies,  cognitive  complexity  in  any  version  is  a  content-free  variable;  it  focuses 
not  on  what  people  think,  but  on  how  they  think.  Thus,  any  position  on  any  topic  can  be  held  at 
any  level  of  complexity.  This  is  not  to  deny  that  in  practice  there  are  fairly  reliable  correlations 
between  some  ideological  positions  and  complexity:  in  general,  convictions  occupying  extremes 
on  an  attitudinal  dimension  tend  to  be  held  more  dogmatically  than  those  nearer  the  center,  and 
therefore  to  be  associated  with  lower  cognitive  complexity.  But  the  relationship  is  with  how  the 
views  are  held,  not  with  their  content;  highly  complex  arguments  can  underlie  some  people’s 
extremist  views  in  politics,  religion,  economics,  science,  etc.  and,  just  as  easily,  simple  ones  can 
support  centrist  views. 

Among  the  best-known  cognitive  style  approaches  is  the  conceptual  complexity  theory  of 
Schroder,  Driver,  and  Streufert  (1967).  That  theory  posits  that  conceptual  complexity  is  a  stable 
personality  trait  that  governs  how  people  process  information,  draw  conclusions,  and  make 
decisions.  Actual  information  processing  and  decision  making  may  vary  in  complexity  in 
response  to  environmental  factors  such  as  information  under-  or  overload,  but  trait  complexity  is 
like  trait  intelligence:  it  imposes  a  limit  on  the  extent  to  which  any  individual  can  process 
information  at  a  complex  level. 

Complexity  itself  is  defined  and  measured  in  terms  of  two  components:  differentiation,  the  ability 
to  perceive  alternative  dimensions  in  a  stimulus  object  or  alternative  legitimate  attitudes  toward  it, 
and  integration,  the  ability  to  perceive  relationships  among  the  differentiated  dimensions  or 
attitudes.  Integration,  therefore,  cannot  occur  in  the  absence  of  differentiation. 

Schroder  et  al.  describe  several  ways  in  which  conceptual  complexity  can  be  measured,  but  the 
one  most  frequently  used  by  them  and  others  is  the  Paragraph  Completion  Test  (PCT).  The  PCT 
is  a  semi-projective  technique  in  which  the  respondent  completes  a  topic  sentence  stem  and 
continues  to  write  on  the  topic,  up  to  a  time  limit,  which  is  the  signal  to  do  the  same  with  the  next 
stem.  The  specific  stems  can  vary  (usually  according  to  the  age,  literacy,  etc.  of  the  subject 
sample  or  the  context  of  the  study);  but  generally  there  are  two  stems  tapping  each  of  three 
important  psychosocial  dimensions:  orientations  to  uncertainty,  attitudes  toward  authority,  and 
reactions  to  interpersonal  rejection.  Each  completion  is  scored  on  a  seven-point  scale;  the  six 
scores  are  then  averaged  to  obtain  the  person’s  total  score.  The  seven  levels  of  scoring  are  shown 
in  Table  1. 


Table  1:  Seven  Levels  of  Integrative  Complexity 


Integrative 

Complexity 

Score 

Explanation  of  Score 

Prototypical  Example  of  Score 

Score  of  1 

There  is  no  sign  of  either 
conceptual  differentiation  or 

“Enforced  retirement  at  65  years  is  most 
certainly  beneficial  to  the  workings  of  our 
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integration  at  this  scoring 
level.  The  author  relies, 
without  qualification,  on  a 
simple,  one-dimensional  rule 
for  interpreting  events  or 
making  choices. 

society.  We  must  realize  that  work  is  the  job  of 
the  young,  and  that  the  elderly  should  leave  their 
work  positions  in  order  to  make  room  for  the 
next  generation.  The  elderly  must  resign 
themselves  to  “let  go”  of  their  family 
domination  and  employment  in  order  that  the 
new  generation  may  explore  its  ideas  and 
promote  growth.  Some  would  argue  that  the 
elderly  have  much  to  contribute  because  they 
have  lived  through  so  much,  but  this  is  clearly 
wrong.  With  the  rapid  rate  of  technological 
change,  the  elderly  are  the  least  able  to  adapt  to 
the  new  conditions.” 

Score  of  2 

The  critical  indicator  for  a 
score  of  2  is  the  potential  or 
conditional  acceptance  of 
different  perspectives  or 
dimensions.  The  author  does 
not  explicitly  develop  the 
alternate  dimension  or 
perspective;  nor  is  it 
necessary  that  it  be  explicitly 
stated  or  named.  Simple 
qualification,  without 
elaboration,  is  sufficient 
evidence  for  a  score  of  2. 

Enforced  retirement  at  65  is  probably  beneficial 
to  our  society.  To  be  sure,  it  is  difficult  to  force 
the  resignation  of  long-time  employees  in  order 
to  make  room  for  the  upcoming  generation.  We 
should,  however,  give  highest  priority  to 
creating  opportunities  for  the  new  generation  to 
explore  their  ideas.  This  need  may  be  especially 
critical  in  view  of  the  requirements  of  running  a 
modern  economy.  More  than  ever,  our  country 
appears  to  require  new  ideas  and  fresh 
perspectives. 

Score  of  3 

The  critical  indicator  for  a 
score  of  3  is  the  recognition 
of  alternative  perspectives  or 
different  dimensions,  and  the 
acceptance  of  these  as  being 
relevant,  legitimate, 
justifiable,  valid,  etc. 

Enforced  retirement  has  recently  engaged  the 
interest  of  the  popular  press.  Some  think  that 
the  elderly  should  give  up  their  jobs  in  order  to 
make  room  for  the  younger  generation.  Such  an 
action  would  bring  fresh  vision  to  business  and 
give  our  society  the  new  ideas  we  clearly  need. 

On  the  other  hand,  there  are  those  who  think  that 
the  experience  of  the  elderly  is  potentially  our 
society’s  deepest  resource,  and  that  their  hard- 
won  knowledge  can  help  business  to  adapt  to 
these  changing  times.  Of  course,  we  rarely  see 
either  vision  or  wisdom  in  its  pure  form;  each 
school  of  thought  tends  to  ignore  a  bit  of  the 
other’s  perspective. 

Score  of  4 

The  author  must  indicate  that 
multiple  perspectives  or 
dimensions  exist,  and  also 

For  society  fully  to  benefit  from  its  members’ 
skills  and  to  survive  the  “technological 
revolution”,  a  new  philosophy  must  develop 
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that  they  could  interact. 

which  unites  the  youth  and  the  elderly  of  the 
nation.  The  technological  skills  of  the  young  are 
important  if  we  are  to  keep  pace  with  the  other 
industrialized  countries.  Flowever,  we  have 
important  social  decision-making  ahead  of  us  if 
we  are  to  improve  our  quality  of  life  in  the  long 
term.  Our  future  success  depends  upon  our 
ability  to  realize  the  potential  of  our  citizens, 
whether  they  be  young  or  old. 

Score  of  5 

The  critical  indicator  of  a 
score  of  5  is  that  alternative 
perspectives  or  dimensions 
are  not  only  held  in  focus 
simultaneously  but  also  are 
viewed  interactively.  The 
author  is  able  to  see  that 
multiple  alternatives  are  all  to 
some  degree  legitimate,  and 
combines  them  to  produce  a 
result  that  none  of  the 
alternatives  could  have 
produced  alone. 

For  society  to  benefit  fully  from  its  members’ 
skills  and  to  survive  the  “technological 
revolution”,  a  new  philosophy  must  develop 
which  unites  the  youth  and  elderly  of  the  nation. 
The  technological  skills  of  the  young  are 
important  if  we  are  to  keep  pace  with  the  other 
industrialized  countries.  Flowever,  we  have 
important  social  decision-making  ahead  of  us  if 
we  are  to  improve  our  quality  of  life  in  the  long 
term.  Thus,  we  must  be  concerned  equally  with 
developing  technology  as  with  developing  a  plan 
for  its  use.  It  is  in  this  latter  area  that  the 
elderly,  with  first-hand  knowledge  of  history, 
must  be  consulted.  With  this  philosophy  in 
mind  we  are  faced  with  the  economic  challenge 
of  employing  both  young  and  senior  citizens. 

Score  of  6 

For  a  paragraph  to  be  given  a 
score  of  6,  the  author  must  be 
working  across  several  levels 
of  schemata  and  at  least  one 
of  the  indicators  noted  above 
must  be  explicitly  delineated. 
Thus,  there  may  be  an 
explicitly  presented  global 
overview  with  only  an 
implicit  indication  of  the 
specific  dynamics  of  the 
alternatives.  Conversely, 
there  may  be  explicitly  stated 
details  about  the  dynamic 
interaction  between 
alternatives  and  only  an 
implicit  communication  of 
the  global  overview. 

Their  experiences  with  war  and  depression 
during  the  thirties  created  in  many  members  of 
our  parents’  generation  a  drive  to  create  some 
form  of  security  for  the  future  that  was  not 
available  for  them  to  enjoy  in  earlier  years.  By 
continuously  building  upon  their  gradually 
increasing  assets  while  still  maintaining  the 
conservative  lifestyles  they  had  been  pressed  to 
follow  during  hard  times,  they  created  economic 
stability  for  themselves.  This  economic 
stability,  enjoyed  by  many  approaching  old  age, 
lends  greater  power  to  seniors’  increasingly 
vocal  demands  for  an  improved  quality  of  life 
for  the  elderly.  Their  offspring,  not  having 
faced  the  same  hardships  as  their  parents,  have 
had  opportunity  and  cause  to  be  somewhat 
reflective  about  issues  pertaining  to  the  quality 
of  life  in  general,  including  the  plight  of  the 
elderly. 
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Score  of  7 


1 .  An  overarching  viewpoint 
is  presented,  which  contains 
an  explanation  of  the 
organizing  principles  (e.g., 
temporal,  causal,  theoretical) 
of  the  problem  or  concept. 

2.  There  is  a  discussion  of  the 
ways  in  which  levels  of  the 
problem  or  concept  interact 
and  thus  demonstrate  the 
validity  of  the  overarching 
perspective.  The  description 
of  the  ways  in  which  levels  of 
the  system  interact  must  be 
both  specific  and  dynamic, 
demonstrating  how  each  level 
is  affected  by  the  other. 

While  these  indicators  are 
distinct,  they  are  inextricably 
linked.  The  global  overview 
encompasses  the  components 
of  a  system,  and  in  fact  may 
have  developed  as  a  result  of 
the  author’s  simultaneous 
consideration  of  these  levels 
or  components. 


We  must  view  this  problem  from  the  very  broad 
perspective  which  involves  the  kind  of  society 
we  see  ourselves  to  be  and  the  kind  of  society 
we  strive  to  be.  Do  I  assume  correctly  that  we 
are,  and  that  we  want  to  be,  civilized?  It  may  be 
that  our  heavy  emphasis  on  individualism, 
productivity  and  self-sufficiency  has  resulted  in 
a  view  of  the  elderly  as  basically  useless 
members  of  society.  On  a  larger  scale,  should 
this  opinion  prevail,  we  could  hardly  have  a 
view  of  ourselves  as  belonging  to  an  integrated 
and  harmonious  community.  At  a  more  specific 
level  of  intergroup  relations,  if  this  attitude 
flourishes,  the  older  segment  of  the  community 
may  feel  forced  to  exert  their  increasing 
economic  and  social  power  to  the  detriment  of 
others  in  society,  such  as  younger  individuals  in 
need  of  their  employment  positions.  This  would 
only  contribute  to  an  adversarial  relationship 
(albeit  reversed),  where  resources  are  still  not 
distributed  with  social  and  economic  equity 
among  a  heterogeneous  population.  We  must  all 
take  our  fair  share  of  responsibility  for  the  state 
we  live  in.  Our  treatment  of  all  individuals, 
regardless  of  age,  depicts  the  state  of  our 
society;  a  civilized  community  treats  all  its 
members  in  a  civilized  manner. 


These  examples  are  drawn  from  Baker-Brown  et  al.  (1992). 
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2  The  Use  of  Complexity  in  Political  Psychology 


Although  the  PCT  has  been  used  in  experimental  research  on  political  attitudes  and  decision¬ 
making  in  politically  relevant  scenarios,  it  has  a  serious  drawback.  As  is  the  case  with  projective 
as  well  as  paper-and-pencil  instruments  as  a  class,  it  requires  the  research  subject  to  be  willing  to 
respond  to  the  tasks  or  questions  in  the  test.  Even  with  current  mechanisms  such  as  web-based 
administration,  which  free  researcher  and  subject  from  having  to  arrange  face-to-face  meetings, 
the  need  for  time  out  from  the  subject’s  normal  daily  activities  in  order  to  participate  remains. 
This  requirement,  unfortunately,  leads  to  some  potential  subjects  of  interest,  such  as  historical 
figures  and  contemporary  high-level  political  leaders,  being  inaccessible;  perhaps  even  worse,  it 
severely  diminishes  the  external  or  ecological  validity  of  the  study.  People  who  agree  to 
participate  in  research  that  they  know  has  the  goal  of  uncovering  their  attitudes,  beliefs,  ideology, 
thinking  processes,  rationale  for  particular  decisions,  and  so  on,  may  give  answers  contaminated 
by  motives  such  as  self-enhancement,  conformity,  desire  to  please  the  researcher,  or  lack  of  self¬ 
insight. 

The  study  of  political  leaders  at  a  distance  overcomes  these  handicaps  by  using  as  its  databases 
information  available  in  archives,  and  materials  produced  by  the  study  subjects  in  the  course  of 
their  ordinary  activities.  With  such  methods,  neither  the  agreement  of  the  subject  to  participate 
nor  subject  expectancy/bias  is  an  issue.  Not  surprisingly,  researchers  interested  in  cognitive 
styles  and  other  aspects  of  information  processing  have  addressed  themselves  to  developing  ways 
in  which  such  variables  can  be  studied  at  a  distance  and  in  a  reliable  manner. 

In  applications  to  political  psychology,  the  study  of  cognitive  complexity  has  branched  in  two 
major  directions.  Both  stem  from  the  same  tree,  the  theory  and  measurement  technique  of 
Schroder  et  al.  (1967).  Perhaps  ironically,  the  more  direct  descendant  has  moved  to  a  more 
divergent  role  than  the  less  direct  one.  The  two  variants  have  different  names,  emphases,  and 
scoring  systems.  One,  integrative  complexity,  belongs  to  a  family  of  tracking  procedures;  i.e., 
repeated  measures  are  taken  during  a  particular  time  period  to  monitor  changes  in  the  subjects’ 
level  of  complexity  and  forecast  decisions  in  the  context  of  problems  occurring  during  that 
period.  The  other,  conceptual  complexity,  belongs  to  a  family  of  profiling  procedures,  which  aim 
at  producing  a  personality  profile  that  allows  researchers  and  policy  makers  to  understand  and 
predict  the  subject’s  ideas  and  predispositions  across  situations.  Despite  these  differences, 
because  both  originate  from  the  same  conceptual  starting  point,  prominently  use  the  term 
“complexity”  for  their  core  variable,  and  seek  to  explain  and  predict  leader  behaviour,  the 
question  of  how  much  they  overlap  is  both  important  and  inescapable. 

2.1  Integrative  Complexity 

Integrative  complexity  (IC;  e.g.,  Suedfeld,  Tetlock,  &  Streufert,  1992)  was  developed  by 
collaborators  of  the  Schroder  et  al.  (1967)  group  and  their  students.  The  perspective  is  that 
complexity  is  a  cognitive  process.  This  approach  differs  from  Schroder  et  al.’s  personological 
version  in  the  following  ways: 

1.  Theoretical  Emphasis.  IC  research,  while  acknowledging  that  there  is  a  stable  level  of  trait 
complexity  as  posited  by  Schroder  et  al.,  focuses  on  state  complexity:  the  levels  of 
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differentiation  and  integration  that  characterize  the  subject’s  information  processing  in  a 
particular  situation.  Schroder  and  his  students  (Driver,  1962;  Streufert  &  Swezey,  1986) 
demonstrated  that  information  load  and  feedback  valence  (success  or  failure)  affect  the 
complexity  of  decisions  aside  from  the  underlying  level  of  conceptual  complexity  as 
measured  by  the  PCT;  Suedfeld  and  his  students  (e.g.,  Suedfeld,  Leighton,  &  Conway,  2005) 
have  concentrated  on  how  a  wide  range  of  both  external  and  internal  conditions  (e.g.,  stress, 
fatigue,  time  pressure,  issue  domain,  accountability),  interacting  with  trait  complexity, 
determines  the  level  of  functional  complexity  at  any  given  time. 

Complex  thinking  is  thought  to  be  a  process  that  takes  up  resources  such  as  time,  energy,  and 
attention.  The  cognitive  manager  model  (Suedfeld,  1992)  hypothesizes  that  good  decision 
makers  conserve  such  resources  by  engaging  any  problem  with  only  as  much  complexity  as  is 
merited  by  its  importance  and  difficulty,  and  by  the  total  resources  available.  Both 
insufficient  and  excessive  complexity  may  lead  to  poorer  decision  outcomes  and  negative 
implications  for  other  decisions.  Thus,  unlike  most  cognitive  complexity  theories,  IC  theory 
does  not  value  highly  complex  thinking  above  a  simpler  alternative;  which  is  preferable 
depends  on  the  situation. 

2.  Methodology /.  IC  research  uses  the  same  7-point  scale  as  PCT  scoring.  As  with  the  PCT, 
thematic  content  analysis  (TCA)  is  used:  that  is,  scoring  depends  on  the  scorer’s  assessment 
of  the  level  of  differentiation  and  integration  in  each  passage.  The  appearance  of  particular 
words  or  phrases  (“content  flags”)  may  alert  the  scorer  to  the  likelihood  of  a  particular  score, 
particularly  at  the  lowest  levels,  but  can  never  be  used  as  full  justification  of  that  score. 

Databases  of  material  used  in  scoring  IC  are  archival  or  concurrent  materials  produced  by 
individuals  of  interest.  These  materials  can  be  written,  oral,  or  electronically  recorded.  As 
with  the  PCT,  almost  any  connected  verbal  passages  can  be  scored.  Paragraphs  to  be  scored 
are  randomly  identified  from  the  whole  available  data  base  (or  all  available  passages  are 
included);  as  far  as  possible,  identifying  information  regarding  the  source,  issue,  and  time  of 
production  is  removed  and  the  paragraphs  scrambled  in  random  order;  and  the  material  is 
then  scored  by  qualified  scorers  using  a  standard  scoring  manual.  Qualification  as  a  scorer 
requires  passing  a  standard  training  course  in  a  3 -day  workshop  or  on  line,  with  a  reliability 
of  at  least  kappa  =  0.85  with  expert  scorers  on  a  test  set  of  paragraphs.  Research  scorers  were 
not  involved  in  the  preparation  of  the  material  for  scoring.  Between  10%  and  20%  of  the 
paragraphs  are  then  re-scored  independently  by  another  qualified  scorer.  Qualified  scorers 
can  usually  score  2-3  paragraphs  per  minute. 

These  procedures  are  relatively  elaborate  and  time-consuming,  but  they  minimize  the  danger 
of  selective  attention  to  content  and  scorer  bias,  and  make  it  possible  to  measure  interscorer 
reliability  and  to  use  standard  significance  tests  and  power  calculations  to  analyze  the  data. 
Regrettably,  several  attempts  by  experts  to  develop  computerized  IC  scoring  systems  have 
failed. 

3.  Research  Goals.  IC  research  does  not  attempt  or  claim  to  assess  leader  personality  or 
cognitive  style.  Rather,  its  aim  is  to  understand  and  predict  decisions  made  by  leaders  in 
specific  situations,  such  as  international  confrontations.  Repeated  sampling  of  leader 
speeches  and  writings  during  the  course  of  such  an  event  is  used  to  track  changes  in  IC, 
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which  in  turn  are  considered  to  be  indicators  of  the  source’s  information  processing  at  each 
data  point  and  to  predict  the  decisions  that  will  emerge. 

Although  research  topics  and  results  cover  a  wide  range,  one  of  the  most  firmly  established  is 
the  finding  that  the  resolution  of  international  confrontations  can  be  predicted  from  the 
changing  IC  patterns  of  major  national  leaders  or  leadership  groups.  Levels  of  IC  that  remain 
steady  or  rise  during  negotiations  precede  peaceful  compromises  at  the  end  of  the  process; 
substantial  drops  in  IC  are  a  sign  that  the  crisis  is  likely  to  lead  to  war  (for  a  review,  see 
Conway,  Suedfeld,  &  Tetlock,  2001). 

The  cognitive  manager  model  (Suedfeld,  1 992)  proposes  that  IC  changes  under  stress  follow 
a  curvilinear  pattern:  when  a  problem  first  arises  that  cannot  be  solved  in  a  simple  way,  IC 
rises  as  the  individual  increases  information  search  and  processing  to  deal  with  it. 
Eventually,  an  asymptote  is  reached;  but  if  the  problem  becomes  too  severe  and  as  the  solver 
exhausts  his  or  her  cognitive  resources,  IC  drops  (“disruptive  stress”)  and  a  simple  strategy  is 
sought  to  escape  the  continued  demands.  In  international  crises,  for  example,  that  simple 
strategy  may  be  to  attack  the  opponent. 

2.2  Conceptual  Complexity 

The  version  of  conceptual  complexity  (CC)  that  is  best  known  in  political  psychology  (e.g., 

Hermann,  1999/2002)  is  a  personological  construct  that  uses  the  same  name  as  the  construct  of 

Schroder  et  al.  (1967),  but  differs  from  the  original  in  several  important  ways. 

1.  Theoretical  Emphasis.  Hermann’s  definition  of  CC  is  entirely  in  terms  of  differentiation. 
The  focus  is  on  specific  words  that  show  a  perception  of  different  dimensions  in  the 
environment.  The  measurement  technique  (see  below)  does  not  allow  for  an  assessment  of 
integration.  CC  scores  are  computed  as  part  of  a  multidimensional  measurement  of  several 
trait  characteristics,  from  which  a  particular  leader’s  stable  personality  is  inferred 
(“Leadership  Trait  Analysis,”  Hermann,  1999/2002).  Because  personality  traits  by  definition 
are  stable  characteristics,  CC  researchers  generally  develop  one  profile  for  each  subject  rather 
than  tracking  possible  changes  over  time  (e.g.,  Dyson,  2009;  Thies,  2009). 

2.  Measurement.  Although  the  scoring  was  originally  done  by  hand,  as  in  the  case  of  IC, 
currently  the  material  taken  from  archived  speeches  or  writings  is  analyzed  by  a  software 
package  (Profiler  Plus),  with  a  recommended  minimum  of  5,000  words  (in  practice,  the 
samples  range  into  the  millions).  This  is  a  much  larger  sample  size  than  is  customary  in  IC 
scoring. 

Like  other  content  analysis  programs,  Profiler  Plus  uses  a  dictionary  developed  by  the 
research  team.  The  program  counts  how  often  a  passage  mentions  words  that  according  to 
the  dictionary  indicate  either  high  or  low  complexity.  High  CC  is  indicated  by  the  more 
frequent  appearance  of  words  such  as  “possibly,”  and  “approximately;”  low  CC,  by  such 
words  as  “certainly”  and  “absolutely.”  Technically,  scores  can  range  from  .00  (no  word  or 
phrase  in  the  material  reflects  high  complexity)  to  1.00  (every  word  reflects  high  complexity). 
Higher  scores  indicate  higher  CC.  Because  all  scoring  is  computerized  once  the  dictionary  is 
prepared  and  entered  into  the  program,  the  method  is  inexpensive  and  fast. 
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3.  Research  Goals.  The  goal  of  profiling  research  is  to  develop  an  accurate  picture  of  the 
personality  of  the  individual  being  studied.  His  or  her  general  orientation  to  life  and  to  the 
leader  role,  perceptions  of  the  international  system,  willingness  to  negotiate  or,  alternatively, 
to  use  armed  force,  personal  motivations  and  goals,  and  similar  variables,  are  measured.  In 
any  particular  application,  the  researcher  must  weigh  these  various  characteristics  to  infer 
their  relative  importance  and  to  predict  the  decisions  that  the  person  is  likely  to  make.  In 
combination  with  self-confidence,  the  CC  score  is  considered  to  be  a  measure  of  openness  to 
information;  whether  it  is  higher  or  lower  than  self-confidence  (which  Profiler  Plus  also 
measures)  differentiates  pragmatic,  democratic  leaders  from  ideologues. 

2.3  Goals  of  the  Current  Study 

The  current  study  was  designed  to  investigate  whether  IC  and  CC  scoring  would  yield  similar 
results  when  applied  to  same  body  of  materials.  The  key  question  is  whether  tracking  and 
profiling  methods  would  be  equally  accurate  in  predicting  a  leader’s  decisions  in  a  particular 
situation  -  i.e.,  whether  the  assessment  of  trait  complexity,  a  stable  long-duration  characteristic  — 
would  suffice  for  such  predictions,  or  whether  state  complexity,  measuring  cognitive  processing 
in  a  situation-specific,  time-limited  way  would  significantly  improve  forecasting. 

In  addition,  an  important  implication  for  researchers  was  at  stake.  As  noted  previously,  IC  scoring 
is  relatively  labour-intensive  and  time-consuming,  both  in  training  scorers  and  in  their  actual 
work.  The  interpretation  of  some  passages  can  require  nuanced  analysis  and  may  lead  to 
disagreement  between  scorers.  CC  scoring,  by  contrast,  has  all  of  the  advantages  of  computer 
analysis:  rapidity,  low  time  requirements,  and  clarity.  If  the  scores  obtained  by  the  two  methods 
were  closely  similar,  considerable  time  and  work  could  be  saved  by  substituting  computer  scoring 
for  the  current  human  scoring  of  IC. 


DRDC  Toronto  CR  2010-022 


9 


3  Method  and  Results 


Several  samples  of  material  were  scored  by  both  methods.  These  included  passages  previously 
scored  as  part  of  IC  studies  in  Peter  Suedfeld’s  laboratory  and  others  previously  scored  in  the 
course  of  CC  studies  in  Margaret  Hermann’s  laboratory.  All  of  the  correlations  were  computed  in 
the  former.  Passages  that  had  been  scored  for  CC  were  scored  for  IC  following  the  standard 
procedures  for  such  scoring;  the  agreement  between  the  two  qualified  IC  scorers  on  this  material 
was  above  85%,  the  level  required  for  reliable  scoring. 

•  Sample  1.  From  examples  in  the  IC  scoring  manual  (Baker-Brown  et  al.,  1992)  and  from 
the  paragraphs  used  in  training  scorers,  for  all  of  which  there  are  scores  assigned  by  a 
consensus  of  experts,  223  passages  were  extracted.  The  passages  were  processed  using  the 
Profiler  Plus  software,  and  Pearson  product-moment  correlations  were  calculated. 

The  overall  correlation  barely  missed  statistical  significance,  r(210)  =  0.13,  p  <  .06.  The 
next  step  was  to  compare  the  correlation  between  the  two  coding  systems  separately  for 
extracts  coded  as  low  vs.  high  in  complexity  according  to  the  IC  scoring  system.  The  former 
category  included  135  extracts  that  had  been  scored  as  1,  2,  or  3  (ranging  from  no 
differentiation  to  differentiation  with  no  integration);  the  latter  included  88  extracts  coded  as 
4,  5,  6,  or  7  (ranging  from  implicit  evidence  of  integration  to  high  level  integration  within  a 
superordinate  schema). 

The  correlation  between  the  two  scoring  systems  for  low  complexity  passages  was  positive 
and  significant,  r(125)  =  0.30,  p  <  .05,  while  that  for  the  high  complexity  extracts  was 
essentially  zero,  r(85)  =  0.01,  NS. 

•  Sample  2.  The  second  part  of  the  study  involved  passages  that  had  been  previously  used  in 
research  by  the  research  group  of  Margaret  Hermann.  The  dataset  included  1 3  speeches 
(244  paragraphs)  by  Muqtada  al-Sadr,  the  leader  of  the  Sadrist  Movement  and  Mahdi  Army, 
the  violent  Shi ’a  resistance  to  the  Coalition  and  newly  elected  government  in  Iraq,  and  12 
speeches  (420  paragraphs)  by  Sayyid  Hassan  Nazrallah,  the  leader  (Secretary  General)  of 
Hezbollah  since  1992.  These  passages  were  then  scored  by  trained  scorers  using  the  IC 
scoring  system,  following  the  usual  procedures.  Reliability  between  the  primary  and 
secondary  scorers  was  87%. 

The  overall  correlation  was  r  =  0.123  (p  <  0.001)  between  the  two  scoring  methods.  In 
order  to  get  a  finer-grain  disaggregated  analysis  than  with  Sample  1,  separate  correlations 
were  calculated  for  passages  scored  as  IC  1  or  2  (no  clear  differentiation),  3  and  4  (clear 
differentiation  only  or  clear  differentiation  with  implied  integration),  and  5-7  (clear 
integration).  The  respective  Pearson  product-moment  correlations  were  r  =  0.1 1 1  (p  <  .02), 
0.038  (NS),  and  0.016  (NS). 

•  Sample  3.  It  may  be  argued  that  because  of  the  prevalence  of  relatively  low-complexity 
materials  in  many  (although  by  no  means  all)  datasets,  the  lack  of  significant  correlations  at 
the  higher  end  of  IC  do  not  really  matter.  If  that  were  the  case,  Profiler  Plus  might  be 
adequate  for  tracking  the  course  of  an  event  and  forecasting  its  outcome  despite  its  omission 
of  integration  as  a  component  of  complexity. 

To  test  this  possibility,  we  used  data  from  a  recent  study  (Frisch,  2009)  that  assessed  IC  at 
various  points  relevant  to  the  onset  and  outcome  of  the  South  Ossetia  (Russo-Georgia)  War 
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1C  Score  (scale  does  not  start  at  zero) 


of  2008.  The  entire  set  of  speeches  and  interviews  of  the  Georgian  president,  Mikheil 
Saakashvili,  from  April  through  September,  a  total  of  1,255  paragraphs  from  49  documents, 
was  scored.  Four  nodal  stages  were  identified:  Baseline  (April  5-19);  the  Russian  downing 
of  a  Georgian  reconnaissance  drone  and  the  growing  hostility  arising  from  that  event  (April 
20-May  18);  negotiations  to  try  to  resolve  the  issue  peacefully,  with  international  mediation 
(May  19- August  6);  and  the  increasing  tension  leading  to  the  war  itself,  beginning  with  the 
Russian  invasion  of  South  Ossetia  and  Abkhazia,  and  ultimately  including  fighting  over 
previously  undisputed  sections  of  Georgia  (August  7-September  31). 

Figure  1  shows  the  IC  and  CC  score  curves  during  the  entire  period.  IC  scoring  followed 
the  usual  pattern:  steep  decreases  as  tension  increased,  higher  IC  when  negotiations  were 
proceeding,  and  a  steep  and  statistically  significant  drop  during  the  period  of  armed  conflict. 
Normally,  we  might  have  expected  an  increase  in  IC  when  the  fighting  stopped,  but  in  fact 
tensions  remained  high  as  Russian  forces  continued  to  occupy  both  breakaway  provinces 
and  the  other  areas  they  had  invaded.  Profiler  Plus  CC  scores  showed  a  significant  increase 
from  baseline  through  negotiations,  not  reflecting  the  initial  period  of  tension,  and  a  non¬ 
significant  drop  at  the  final  stage. 
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Figure  1:  IC  and  CC  Scores  for  Mikheil  Saakashvili 
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4  Discussion 


Both  data  analyses  indicate  that  the  across-the-board  correlation  between  IC  hand-scoring  and  CC 
scoring  using  Profiler  Plus  is  quite  low,  reaching  significance  as  a  function  of  the  large  sample 
size.  That  overall  correlation  level  is  driven  by  the  agreement  between  the  two  systems  on 
identifying  the  lowest  complexity:  i.e.,  passages  that  do  not  even  reach  the  level  of  differentiation. 
For  passages  in  the  differentiated  range,  the  correlation  fell  below  .04,  and  in  the  integrated  range, 
it  was  essentially  zero  in  both  samples. 

The  fundamental  theory  of  IC  ensures  that  the  measure  is  not  useful  for  personality  profiling;  in 
that  context,  the  low  correlations  do  not  matter.  The  question  of  whether  CC  scoring  is  useful  for 
tracking  is  also  somewhat  moot,  as  in  practice  CC  researchers  consider  that  they  are  studying  a 
stable  personality  factor  so  that  repeated  measurement  within  a  short  time  period  is  unnecessary 
and  useless. 

The  low  level  of  compatibility  between  the  two  systems  may  be  somewhat  ameliorated  by  the  fact 
that  in  many  IC  studies,  the  mean  score  of  paragraphs  is  at  the  1  -2  level.  However,  that  mean  is 
the  product  of  undifferentiated  passages  being  averaged  with  scores  above  that  level.  Many 
passages  and  complete  speeches  or  writings  reach  the  level  of  clear  differentiation,  and  some  go 
considerably  higher.  To  ignore  the  levels  of  clear  differentiation  to  high-level  integration  would 
be  to  miss  important  aspects  of  thinking  and  decision-making.  It  would  also  make  it  impossible, 
or  at  least  very  difficult,  to  identify  the  onset  and  impact  of  disruptive  stress,  which  presages  a 
simplifying,  often  violent  outcome. 

As  a  last  point  on  this  issue,  using  CC  as  a  tracking  rather  than  profiling  tool  did  not  work  out 
well  in  the  case  of  the  Russo-Georgian  War.  Thus,  theoretical,  methodological,  and  empirical 
arguments  all  point  to  the  conclusion  that  neither  method  can  be  used  to  substitute  for  the  other. 

Our  conclusion  has  to  be  that  both  methods  are  appropriate  for  their  respective  purposes;  but, 
regrettably,  the  computerized  scoring  of  integrative  complexity  is  not  yet  on  the  horizon. 
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cc 

Conceptual  Complexity 

IC 

Integrative  Complexity 

PCT 

Paragraph  Completion  Test 

RWA 

Right-Wing  Authoritarianism 

TCA 

Thematic  Content  Analysis 
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(U)  Measures  of  the  cognitive  complexity  of  leaders  have  been  used  to  infer  the  flexibility, 
open-endedness,  and  information-orientation  of  their  decision-making  in  international 
and  non-state  confrontations.  At  present,  there  are  two  major  methods  of  “assessment  at 
a  distance”  used  in  this  context.  One  uses  computer  scoring  to  develop  personality  profiles 
of  leaders;  the  other  uses  a  more  labour-  and  time-intensive  human  scoring  system  to 
track  changes  in  the  target’s  thinking  to  predict  the  outcome  of  a  particular  confrontation.  If 
computer  scoring  were  able  to  make  event-specific  predictions,  the  saving  in  time  and 
work  would  be  substantial.  This  study  compared  the  two  systems  to  establish  (a)  whether 
the  computer-scored  system  could  replace  human  scoring,  and  (b)  using  the  example  of 
the  South  Ossetia  War  between  Georgia  and  Russia,  which  method  was  a  better  predictor 
of  rising  and  falling  tension,  The  data  confirmed  the  relevance  of  integrative  complexity 
measurement  in  a  new  context,  that  of  an  ongoing  confrontation  with  changing  levels  of 
tension,  up  to  and  including  war,  between  a  major  and  a  minor  national  power. 

The  correlation  between  scores  from  the  two  methods  was  low;  at  high  levels  of  cognitive 
complexity,  it  was  essentially  zero.  The  human  scoring  of  integrative  complexity,  which 
tracks  changes  in  complexity  over  the  duration  of  a  particular  event,  was  closely  tied  to  the 
course  of  the  confrontation;  the  computer  scoring  of  cognitive  complexity,  which  profiles 
complexity  as  a  stable  personality  characteristic,  was  not.  Thus,  although  computer 
scoring  has  significant  advantages  in  cost  and  time,  it  does  not  accomplish  the  same 
goals.  This  finding  has  important  implications  for  security/intelligence  applications,  as  the 
computer  scoring  approach  is  an  attractive  way  to  process  large  amounts  of  information. 
However,  our  data  indicate  that  the  negative  trade-off  between  speed  and  accuracy  is 
serious  enough  to  opt  for  the  more  laborious  human  tasking  if  the  goal  is  the  prediction  of 
crisis  outcome. 

(U)  Nous  avons  eu  recours  a  des  mesures  de  la  complexite  cognitive  chez  les  leaders  pour 
prevoir  le  caractere  conciliant,  ouvert  et  axe  sur  I’information  du  processus  de  prise  de 
decision  dans  les  situations  d’affrontement  internationales  et  non  etatiques.  A  I’heure 
actuelle,  on  utilise  principalement  deux  methodes  «  devaluation  a  distance  »  dans  ce 
contexte.  La  premiere  etablit  un  profil  de  la  personnalite  des  chefs  en  ayant  recours  a  la 
cotation  informatisee;  la  deuxieme  fait  appel  a  une  methode  de  cotation  par  des  humains 
plus  exigeante  en  main-d’oeuvre  et  en  temps  pour  effectuer  un  suivi  des  changements 
dans  le  raisonnement  du  sujet  cible  afin  de  prevoir  Tissue  d’un  affrontement  en  particulier. 
Si  la  cotation  informatisee  permettait  de  faire  des  previsions  relativement  a  un  evenement 
precis,  Teconomie  de  temps  et  de  travail  ainsi  realisee  serait  considerable.  Dans  le  cadre 
de  la  presente  etude,  nous  avons  compare  les  deux  systemes  dans  le  but  de  a) 
determiner  si  le  systeme  de  cotation  informatise  pouvait  remplacer  le  systeme  de  cotation 
par  des  humains;  b)  determiner  quelle  methode  permettait  le  mieux  de  prevoir 
Texacerbation  et  le  relachement  de  la  tension,  a  I’aide  de  Texemple  du  conflit  Georgie 
Russie  en  Ossetie  du  Sud.  Les  donnees  ont  confirme  la  pertinence  de  la  mesure  de  la 
complexite  integrative  dans  un  contexte  nouveau,  celui  d’un  affrontement  en  cours  entre 
une  puissance  nationale  majeure  et  une  puissance  nationale  mineure,  affrontement 
caracterise  par  un  niveau  de  tension  changeant  qui  peut  atteindre  et  comprendre  la 
guerre. 

La  correlation  entre  les  scores  obtenus  a  I’aide  des  deux  methodes  etait  faible;  a  des 
niveaux  eleves  de  complexite  cognitive,  elle  etait  essentiellement  egale  a  zero.  La  cotation 
de  la  complexite  integrative  par  des  humains,  qui  permet  de  suivre  les  changements  sur  le 


plan  de  la  complexite  pendant  le  deroulement  d’un  evenement  donne,  correspondait 
etroitement  a  revolution  de  I’affrontement;  ce  n’etait  pas  le  cas  de  la  cotation  informatisee 
de  la  complexite  cognitive,  qui  presente  la  complexite  comme  un  trait  de  personnalite 
stable.  Par  consequent,  meme  si  la  cotation  informatisee  presente  de  nombreux 
avantages  sur  le  plan  des  couts  et  du  temps  requis,  elle  ne  permet  pas  d’atteindre  les 
memes  objectifs.  Ce  constat  a  des  incidences  importantes  sur  les  applications  en  matiere 
de  securite/renseignements,  etant  donne  que  la  cotation  informatisee  est  une  faqon 
commode  de  traiter  un  volume  important  de  donnees.  Neanmoins,  nos  resultats  indiquent 
que  le  gain  de  rapidite  est  accompagne  d’une  diminution  de  I’exactitude  suffisamment 
importante  pour  justifier  le  recours  a  la  methode  plus  laborieuse  faisant  appel  aux 
humains  lorsque  I’objectif  est  de  prevoir  I’issue  d’une  crise. 
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